1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing method for expanding and variably multiplying input video information or resolution-converting input low-resolution information into high-resolution information.
2. Related Background Art
Various methods for resolution-converting input low-resolution information into high-resolution information have been proposed so far. The proposed conventional methods are different in conversion method depending on the type of a purposed image (for example, multivalued image having gradation information for each pixel, binary image converted into binary notation by a pseudo intermediate gradation, binary image converted into binary notation by fixed threshold, or character image). One of the following methods is generally used as a conventional interpolation method: a closest interpolation method of arranging the same pixel values closest to the interpolation point shown in FIG. 20 and a common primary interpolation method of determining a pixel value E through the following calculation by using four points (pixel values of four points are assumed as A, B, C and D) shown in FIG. 21.
E=(1xe2x88x92i)(1xe2x88x92j)A+ixc2x7(1xe2x88x92j)B+jxc2x7(1xe2x88x92i)C+ijD
(In the above expression, when setting an inter-pixel distance to 1, it is assumed that there is a distance i from A in the transverse direction and a distance j from A in the longitudinal direction., ixe2x89xa61, jxe2x89xa61)
However, the above conventional example has the following disadvantages.
First, though the method shown in FIG. 20 has an advantage that the structure is simple, blocks are visually conspicuous and image quality is inferior because a pixel value is determined for each expanding block when using a purposed image for a natural image.
Moreover, when using a purposed image for a character, line image, or CG (computer graphic) image, the same pixel value continues for each block. Therefore, particularly, a diagonal results in a remarkably-notched inferior image referred to as jaggies. FIGS. 22 and 23 show a state in which a jaggies occurs. FIG. 22 shows input information and FIG. 23 shows an example of resolution conversion in which the number of pixels is doubled in longitudinal and transverse directions in accordance with the method in FIG. 20. In general, image quality further deteriorates as a magnification increases. (Numerals xe2x80x9c200xe2x80x9d and xe2x80x9c10xe2x80x9d in FIG. 23 denote pixel values.)
The method in FIG. 21 is a method generally frequently used for expansion of a natural image. In the case of this method, though an averaged and smoothed image quality is obtained, an edged portion or a portion requiring a sharp image quality has a dull image quality. Moreover, in the case of an image obtained by scanning a map or a natural image including characters, important information may not be communicated to a receiver due to halation caused by interpolation.
FIG. 24 shows video information obtained by doubling the input video information in FIG. 22 in longitudinal and transverse directions.
Also as shown in FIG. 24, not only the circumference of a diagonal but also the diagonal cause halation because pixel values are not uniform.
Therefore, various arts for performing resolution conversion at a high image quality have been proposed so far. For example, U.S. Pat. No. 5,280,546 discloses an interpolation method of separating a natural image from a line image in order to remove interpolation halation, applying linear interpolation to the natural image and arranging the maximum and minimum values of peripheral pixels on the linear image by converting the linear interpolation into binary notation. In the case of this art, however, occurrence of jaggies cannot be avoided because interpolation halation occurs on the natural image and high-resolution information is generated while the resolution of low-resolution information is kept in the line image. Moreover, because the line image is also quantized to two gradations, it cannot be avoided that a multigradation line image has an image quality with fewer gradations though there is no problem when original information has only two gradations.
U.S. Pat. No. 5,430,811 shows a non-linear interpolation art using LUT. In the case of this art, however, an algorithm can correspond only to a magnification of doublenessxc3x97doubleness. Therefore, to obtain other magnification, it is necessary to repeat the above processing or use the processing together with other expansion processing and thus, processing becomes complex. Moreover, even if a magnification of 2n is realized in repetitive processing, it is not easy to perform control so as to provide a desired non-linearity for each pixel of high-resolution information in the final magnification. Furthermore, in the case of this art, occurrence of jaggies cannot be avoided because values of observation pixels (original information) are not changed similarly to the case of U.S. Pat. No. 5,280,546.
Furthermore, U.S. Pat. No. 5,054,100 also discloses non-linear interpolation. However, though this art is effective for simple interpolation halation of an edge in longitudinal and transverse directions, an effect of the art cannot be obtained if video information becomes slightly complex. Occurrence of jaggies cannot be avoided similarly to the case of the above two conventional examples. Arts for realizing preferable resolution conversion by such as dither method and error diffusion method have been known from old times, in which linear interpolation and reconversion-into-binary-notation are applied to the resolution conversion of a binary image provided with pseudo gradation processing after passing a digital filter through the vicinity of a noticeable pixel and converting the binary image into a multivalued image (U.S. Pat. No. 4,803,558, Japanese Patent Publication No. 61-56665, and the like). There is a proposal of performing preferable resolution conversion when an original image is a multivalued image by using the above arts. However, it is difficult to form a preferable multivalued image provided with spatial continuity of pixel values as long as a quantization art such as a conversion-into-binary-notation art is used as a system for forming an edge.
To remove the above-described disadvantages of the prior art, the present applicant disclosed a conversion preferable for image quality free from jaggies without causing halation due to interpolation which was a particular problem on a natural image or without depending on the low resolution of input original information in U.S. Pat. No. 5,875,268, Japanese Patent Application Laid-Open No. 9-252400, Japanese Patent Application Laid-Open No. 10-248012, and the like.
In the case of the art disclosed in the above official gazettes, however, because high-resolution information is generated by performing non-linear processing every pixel of low-resolution information, the continuity with high-resolution information generated by adjacent pixels is lost and an edged portion of a natural image may be formed into an edged shape serrated in the gradation direction. Particularly, as a magnification of resolution conversion increases, a natural image in which continuity of pixel values is broken and which is artificial and has a sense of incongruity may be formed.
It is an object of the present invention to provide an image processing apparatus and an image processing method capable of realizing a preferable conversion in image quality without causing halation due to interpolation when converting input low-resolution information into high-resolution information, depending on a low resolution of input original information, or causing no jaggies.
It is another object of the present invention to provide an image processing apparatus and an image processing method making it possible to generate preferable high-resolution information in which continuity is not broken even in a conversion having any non-linearity because it is possible to extinguish the discontinuity at a boundary point every conversion in conversions having non-linearity.
To achieve the above objects, an image processing apparatus of the present invention uses an image processing apparatus for inputting low-resolution video information, expanding one pixel to Nxc3x97M pixels (each of N and M is an integer equal to or larger than 1 but a case of N=1 and M=1 is excluded), and resolution-converting the input image into a high-resolution image, comprising:
first window formation means for referring to peripheral pixels including noticeable pixels;
a plurality of conversion means different in conversion method of converting one pixel into Nxc3x97M pixels; and
selection means for selecting a proper conversion means out of the above plurality of conversion means in accordance with a distribution state of pixel values in a first window; wherein
the conversion means evaluate the type of a selected conversion means in adjacent pixels including noticeable pixels and calculate conversion values for Nxc3x97M pixels of the noticeable pixels in accordance with the evaluation result.
The image processing apparatus of the present invention further preferably comprises:
second window formation means for referring to the type of a selected conversion means in adjacent pixels including noticeable pixels;
evaluation means for evaluating combinations in a second window;
setting means for setting a representative point at the boundary between a noticeable pixel and an adjacent pixel in accordance with an evaluation result by the evaluation means; and
representative-point conversion means for calculating a conversion value of the representative point in accordance with a conversion condition of a selected conversion means of an adjacent pixel; wherein
the conversion means calculates conversion values of Nxc3x97M pixels of noticeable pixels in accordance with the conversion value of the representative point.
Moreover, an image processing method of the present invention uses an image processing method for inputting low-resolution video information, expanding one pixel to Nxc3x97M pixels (each of N and M is an integer equal to or larger than 1 but a case of N=1 and M=1 is excluded), and resolution-converting the input image into a high-resolution image, comprising:
a first window-forming step of referring to peripheral pixels including noticeable pixels;
a plurality of conversion steps different in conversion method of converting one pixel into Nxc3x97M pixels; and
a selection step of selecting a proper conversion step out of the above plurality of conversion steps in accordance with a distribution state of pixel values in a first window; wherein
the conversion steps evaluate the type of a selected conversion step in adjacent pixels including noticeable pixels and calculate conversion values for Nxc3x97M pixels of the noticeable pixels in accordance with the evaluation result.
The image processing method of the present invention further preferably comprises:
a second window-forming step of referring to the type of a selected conversion step in adjacent pixels including noticeable pixels;
an evaluation step of evaluating combinations in a second window;
a setting step of setting a representative point at the boundary between a noticeable pixel and an adjacent pixel in accordance with an evaluation result by the evaluation step; and
representative-point conversion step of calculating a conversion value of the representative point in accordance with a conversion condition of a selected conversion step of an adjacent pixel; wherein the conversion step calculates conversion values of Nxc3x97M pixels of noticeable pixels in accordance with the conversion value of the representative point.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.